Correcting the Minimization Bias in Searches for Small Signals

We discuss a method for correcting the bias in the limits for small signals if those limits were found based on cuts that were chosen by minimizing a criterion such as sensitivity. Such a bias is commonly present when a "minimization" and an "evaluation" are done at the same time. We propose to use a variant of the bootstrap to adjust the limits. A Monte Carlo study shows that these new limits have correct coverage.Comment: 14 pages, 5 figue

Some Aspects of Measurement Error in Linear Regression of Astronomical Data

I describe a Bayesian method to account for measurement errors in linear regression of astronomical data. The method allows for heteroscedastic and possibly correlated measurement errors, and intrinsic scatter in the regression relationship. The method is based on deriving a likelihood function for the measured data, and I focus on the case when the intrinsic distribution of the independent variables can be approximated using a mixture of Gaussians. I generalize the method to incorporate multiple independent variables, non-detections, and selection effects (e.g., Malmquist bias). A Gibbs sampler is described for simulating random draws from the probability distribution of the parameters, given the observed data. I use simulation to compare the method with other common estimators. The simulations illustrate that the Gaussian mixture model outperforms other common estimators and can effectively give constraints on the regression parameters, even when the measurement errors dominate the observed scatter, source detection fraction is low, or the intrinsic distribution of the independent variables is not a mixture of Gaussians. I conclude by using this method to fit the X-ray spectral slope as a function of Eddington ratio using a sample of 39 z < 0.8 radio-quiet quasars. I confirm the correlation seen by other authors between the radio-quiet quasar X-ray spectral slope and the Eddington ratio, where the X-ray spectral slope softens as the Eddington ratio increases.Comment: 39 pages, 11 figures, 1 table, accepted by ApJ. IDL routines (linmix_err.pro) for performing the Markov Chain Monte Carlo are available at the IDL astronomy user's library, http://idlastro.gsfc.nasa.gov/homepage.htm

Nonparametric Regression using the Concept of Minimum Energy

It has recently been shown that an unbinned distance-based statistic, the energy, can be used to construct an extremely powerful nonparametric multivariate two sample goodness-of-fit test. An extension to this method that makes it possible to perform nonparametric regression using multiple multivariate data sets is presented in this paper. The technique, which is based on the concept of minimizing the energy of the system, permits determination of parameters of interest without the need for parametric expressions of the parent distributions of the data sets. The application and performance of this new method is discussed in the context of some simple example analyses.Comment: 10 pages, 4 figure

Critical behavior of the three-dimensional bond-diluted Ising spin glass: Finite-size scaling functions and Universality

We study the three-dimensional (3D) bond-diluted Edwards-Anderson (EA) model with binary interactions at a bond occupation of 45% by Monte Carlo (MC) simulations. Using an efficient cluster MC algorithm we are able to determine the universal finite-size scaling (FSS) functions and the critical exponents with high statistical accuracy. We observe small corrections to scaling for the measured observables. The critical quantities and the FSS functions indicate clearly that the bond-diluted model for dilutions above the critical dilution p*, at which a spin glass (SG) phase appears, lies in the same universality class as the 3D undiluted EA model with binary interactions. A comparison with the FSS functions of the 3D site-diluted EA model with Gaussian interactions at a site occupation of 62.5% gives very strong evidence for the universality of the SG transition in the 3D EA model.Comment: Revised version. 10 pages, 9 figures, 2 table

A comparison of block and semi-parametric bootstrap methods for variance estimation in spatial statistics

Efron (1979) introduced the bootstrap method for independent data but it cannot be easily applied to spatial data because of their dependency. For spatial data that are correlated in terms of their locations in the underlying space the moving block bootstrap method is usually used to estimate the precision measures of the estimators. The precision of the moving block bootstrap estimators is related to the block size which is difficult to select. In the moving block bootstrap method also the variance estimator is underestimated. In this paper, first the semi-parametric bootstrap is used to estimate the precision measures of estimators in spatial data analysis. In the semi-parametric bootstrap method, we use the estimation of the spatial correlation structure. Then, we compare the semi-parametric bootstrap with a moving block bootstrap for variance estimation of estimators in a simulation study. Finally, we use the semi-parametric bootstrap to analyze the coal-ash data

One-dimensional infinite component vector spin glass with long-range interactions

We investigate zero and finite temperature properties of the one-dimensional spin-glass model for vector spins in the limit of an infinite number m of spin components where the interactions decay with a power, \sigma, of the distance. A diluted version of this model is also studied, but found to deviate significantly from the fully connected model. At zero temperature, defect energies are determined from the difference in ground-state energies between systems with periodic and antiperiodic boundary conditions to determine the dependence of the defect-energy exponent \theta on \sigma. A good fit to this dependence is \theta =3/4-\sigma. This implies that the upper critical value of \sigma is 3/4, corresponding to the lower critical dimension in the d-dimensional short-range version of the model. For finite temperatures the large m saddle-point equations are solved self-consistently which gives access to the correlation function, the order parameter and the spin-glass susceptibility. Special attention is paid to the different forms of finite-size scaling effects below and above the lower critical value, \sigma =5/8, which corresponds to the upper critical dimension 8 of the hypercubic short-range model.Comment: 27 pages, 27 figures, 4 table

Critical behavior of the Random-Field Ising Magnet with long range correlated disorder

We study the correlated-disorder driven zero-temperature phase transition of the Random-Field Ising Magnet using exact numerical ground-state calculations for cubic lattices. We consider correlations of the quenched disorder decaying proportional to r^a, where r is the distance between two lattice sites and a<0. To obtain exact ground states, we use a well established mapping to the graph-theoretical maximum-flow problem, which allows us to study large system sizes of more than two million spins. We use finite-size scaling analyses for values a={-1,-2,-3,-7} to calculate the critical point and the critical exponents characterizing the behavior of the specific heat, magnetization, susceptibility and of the correlation length close to the critical point. We find basically the same critical behavior as for the RFIM with delta-correlated disorder, except for the finite-size exponent of the susceptibility and for the case a=-1, where the results are also compatible with a phase transition at infinitesimal disorder strength. A summary of this work can be found at the papercore database at www.papercore.org.Comment: 9 pages, 13 figure

One-Dimensional Directed Sandpile Models and the Area under a Brownian Curve

We derive the steady state properties of a general directed ``sandpile'' model in one dimension. Using a central limit theorem for dependent random variables we find the precise conditions for the model to belong to the universality class of the Totally Asymmetric Oslo model, thereby identifying a large universality class of directed sandpiles. We map the avalanche size to the area under a Brownian curve with an absorbing boundary at the origin, motivating us to solve this Brownian curve problem. Thus, we are able to determine the moment generating function for the avalanche-size probability in this universality class, explicitly calculating amplitudes of the leading order terms.Comment: 24 pages, 5 figure

Cross-correlations in scaling analyses of phase transitions

Thermal or finite-size scaling analyses of importance sampling Monte Carlo time series in the vicinity of phase transition points often combine different estimates for the same quantity, such as a critical exponent, with the intent to reduce statistical fluctuations. We point out that the origin of such estimates in the same time series results in often pronounced cross-correlations which are usually ignored even in high-precision studies, generically leading to significant underestimation of statistical fluctuations. We suggest to use a simple extension of the conventional analysis taking correlation effects into account, which leads to improved estimators with often substantially reduced statistical fluctuations at almost no extra cost in terms of computation time.Comment: 4 pages, RevTEX4, 3 tables, 1 figur